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PERFORMANCE  OF  AIR  TRAFFIC  CONTROL  SPECIALISTS  (ATCS’S) 

ON  A  LABORATORY  RADAR  MONITORING  TASK:  AN  EXPLORATORY  STUDY 
OF  COMPLACENCY  AND  A  COMPARISON  OF  ATCS  AND  NON-ATCS  PERFORMANCE 


Introduction. 

The  role  of  the  air  traffic  control  specialist  (ATCS)  in  proposed 
highly  automated  air  traffic  systems  of  the  future  Is  currently  receiving 
considerable  attention.  At  the  present  time,  a  prevalent  conception  of 
the  controller's  role  in  such  systems  is  that  of  a  "systems  monitor"  or 
"systems  manager."  Inherent  in  this  view  is  the  belief  that  the  role 
of  the  future  controller  will  be  less  that  of  an  active  planner  and  more 
that  of  a  passive  responder  to  alternative  courses  of  action  presented 
by  the  computer  system.  Such  a  change  in  role  has  raised  concerns 
that  increased  controller  complacency,  inattentiveness,  boredom,  and 
reduced  readiness  to  react  in  emergencies  may  become  serious  problems 
in  the  systems  being  planned  (2) . 

Over  the  past  5  years,  our  laboratory  has  been  engaged  in  a  program 
of  research  to  examine  some  of  these  concerns,  as  mil  as  several  others, 
through  the  use  of  a  simulated  radar  monitoring  task  (5,6,7,8,9,10,11). 

This  task  was  designed  to  incorporate  some  of  the  basic  skills  (e.g., 
sustained  attention,  continuous  scanning  activity,  minimal  information 
processing)  that  might  characterize  future,  highly  automated  air  traffic 
control  (ATC)  systems.  Two  of  the  concerns  noted  previously  (i.e.,  inatten¬ 
tiveness  and  boredom-monotony)  have  already  been  studied  and  found  to  show 
some  relationship  to  performance  decline  on  this  task  (5).  Another,  readi¬ 
ness  to  react  and  the  time  course  of  emotional  recovery  to  an  emergency 
situation,  will  be  addressed  in  a  study  planned  for  next  year. 


Of  all  of  the  concerns  that  have  been  expressed  with  respect  to 


high  levels  of  automation,  the  one  heard  most  frequently  is  that  of 
complacency.  In  the  context  of  automated  systems,  complacency  refers 
to  a  feeling  of  well-being  or  security  engendered  by  a  system  that 
presumably  operates  smoothly,  efficiently,  and  quite  reliably  with  minimal 
controller  input.  It  now  seems  likely  that  future  En  Route  systems  will 
be  able  to  process  and  handle  most  of  the  routine  situations  currently 
handled  by  controllers,  leaving  the  controller  with  the  task  of  simply 
verifying  that  those  options  presented  by  the  computer  are  appropriate 
(2,3).  This  reduced  level  of  controller  involvement  (relative  to  existing 
systems)  could  easily  lead  to  some  major  shifts  in  the  task  of  the  controller 
in  advanced  systems,  and  the  possibility  that  these  shifts  could  result  in 
increased  controller  complacency  seems  to  be  a  realistic  concern. 

The  present  study  was  undertaken  to  explore  the  feasibility  of  one 
technique  for  varying  complacency  and  to  determine  its  effect  on  sustained 
attention.  In  essence,  the  study  compared  experienced  ATCS's  under  one  of 
two  conditions:  (1)  a  condition  in  which  the  subject  took  a  designated 
action  in  order  to  correct  departures  of  displayed  target  altitudes  from 
assigned  limits,  and  (ii)  a  condition  in  which  the  subject  took  corrective 
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action  only  if  the  computer  failed  to  correct  the  altitude  deviation.  It 
was  hypothesized  that  complacency  would  be  greater  under  the  latter  condi¬ 
tion  and  that  this  would  be  reflected  in  a  greater  decline  in  attentiveness 
(Increase  in  detection  times) . 

In  addition  to  studying  complacency,  the  present  study,  by  virtue  of 
the  fact  that  ATCS's  were  used  as  subjects,  allowed  vs  to  compare  the 
general  performance  level  of  experienced  controllers  on  our  radar  monitoring 
task  with  the  patterns  obtained  in  our  previous  studies,  all  of  which  used 
subjects  with  no  prior  experience  or  training  in  air  traffic  control. 

Method. 

A.  Subjects.  Sixteen  volunteer  ATCS  Instructors  (15  men  and  1  woman) 
participated  in  the  experiment.  All  were  from  the  Federal  Aviation  Admin¬ 
istration  Academy  at  the  Mike  Monroney  Aeronautical  Center  in  Oklahoma  City. 
Half  of  the  16  subjects  were  randomly  assigned  to  the  computer-controlled 
condition  and  half  to  the  subject-controlled  condition.  The  respective 
mean  ages  and  years  of  field  experience  of  subjects  assigned  to  these 

two  conditions  were  38.75  years  (age),  10.6  years  (experience),  and  34.75 
years  (age),  12.6  years  (experience).  The  groups  did  not  differ  signi¬ 
ficantly  in  either  age  oi  years  of  experience  (j>  >  .05).  All  were  tested 
and  found  to  have  normal  near-point  visual  acuity  (corrected  to  20/20  if 
necessary) . 

B.  Design  and  Task  Apparatus.  Task  programing  and  recording  of 
responses  were  accomplished  using  a  Digital  Equipment  Corporation  (DEC) 
PDP-11/40  computer.  The  computer  was  interfaced  with  a  VT-11  (DEC)  17-inch 
(43  cm)  cathode-ray  tube  (CRT),  which  served  as  the  subject's  display.  The 
CRT  was  located  in  a  console  resembling  an  air  traffic  control  radar  unit. 
The  stimuli  (targets)  consisted  of  alphanumeric  data  blocks,  each  represen¬ 
ting  an  aircraft.  Data  blocks  comprised  two  rows  of  symbols:  the  top  row, 
consisting  of  two  letters  and  three  numerals,  identified  the  aircraft, 
while  the  bottom  row  of  six  numerals  indicated  its  altitude  and  speed. 

The  first  three  of  these  numerals  gave  altitude  in  hundreds  of  feet  and 
the  last  three  gave  groundspeed. 

A  simulated  radar  sweepline  was  employed  that  made  one  complete  clock¬ 
wise  revolution  every  6  seconds.  A  target  was  updated  as  to  (1)  location 
and  (ii)  any  change  in  its  data  block  moments  after  the  sweepline  passed 
the  target' 8  prior  location.  Sixteen  targets  were  present  at  all  times; 
as  one  left,  another  appeared  on  the  screen.  The  critical  stimulus  or 
signal  to  which  the  subject  was  instructed  to  respond  consisted  of  a 
change  in  a  target's  displayed  altitude  to  a  value  greater  than  550  or 
less  than  150.  The  values  of  the  increases  or  decreases  in  altitude  were 
randomly  determined,  except  that  the  changed  altitude  value  could  not 
be  greater  than  599  or  less  than  100.  Ten  such  critical  stimuli  appeared 
in  each  30-minute  period;  five  occurred  in  the  first  15  minutes  and  five 
in  the  second. 

For  both  subject-  and  computer-controlled  conditions,  the  subject's 
Initial  response  was  the  same  and  consisted  of  pressing  a  button  held  in 


the  right  hand  as  soon  as  a  critical  altitude  change  was  detected.  The 
time  between  stimulus  occurrence  and  this  response  constituted  the  measure 
of  target  detection  time.  In  the  subject-controlle  \  condition,  the  button 
response  was  followed  by  the  act  of  holding  a  light  pen  over  the  critical 
target  which,  in  turn,  caused  the  altitude  to  revert  to  its  previous 
within-limits  value.  If  a  critical  stimulus  was  not  detected  within  60 
seconds,  the  data  block  automatically  reverted  to  its  previous  value.  In 
the  computer-controlled  condition,  pressing  the  response  button  not  only 
recorded  detection  time,  but  also  served  to  activate  a  red  indicator 
light  located  adjacent  to  the  right  side  of  the  screen.  This  light 
remained  on  for  60  seconds.  The  subject  was  instructed  to  use  the  light 
pen  to  correct  the  altitude  departure  only  if  the  altitude  did  not  change 
to  a  within-limits  value  before  the  red  light  went  off.  Since  a  critical 
target  always  timed  out  (reverted  to  its  previous  value)  60  seconds  after 
the  altitude  departure  had  occurred,  the  subject's  variable  detection 
time  plus  the  60-second  time  that  the  red  light  was  activated  ensured 
that  a  target  always  returned  to  a  within-limits  value  prior  to  the  red 
light's  offset.  Target  detection  times  for  both  conditions  were  recorded 
by  the  computer  for  subsequent  processing. 

C.  Procedure.  On  arrival,  the  subject  was  taken  to  the  experimental 
room  and  an  orientation  tape  played.  The  orientation  explained  that  this 
study  was  one  of  a  series  designed  to  investigate  various  aspects  of 
highly  automated  systems.  The  subject  was  told  that  the  role  of  control¬ 
lers  in  future  ATC  systems  would  likely  be  quite  different  from  the  role 
of  present-day  controllers,  and  that  this  experiment  was  being  conducted 
to  examine  the  possible  effects  of  some  of  these  changes.  Following  this, 
a  9-point  subjective  rating  scale  was  administered  dealing  with  present 
feelings  of  attentiveness,  fatigue,  tension,  and  boredom. 

The  instructions  for  the  task  itself  emphasized  the  necessity  of 
pressing  the  button  immediately  upon  detection  of  a  critical  stimulus. 

The  subject  was  told  that  a  critical  stimulus  (any  altitude  value  greater 
than  550  or  less  than  150)  could  occur  in  any  target  at  any  time,  regard¬ 
less  of  the  current  altitude  values  of  the  targets.  It  was  explained 
that  occasional  large  changes  in  altitude  would  not  normally  occur  in 
an  actual  radar  system,  but  that  this  departure  from  normal  conditions 
was  necessary  to  Insure  that  all  targets  would  be  given  equal  priority 
in  scanning. 

In  the  subject-controlled  condition,  he/she  was  told  to  initiate  an 
action  with  the  light  pen  to  correct  the  altitude  departure  after  the 
button  press  had  been  made.  In  the  computer-controlled  condition,  subjects 
were  told  to  take  corrective  action  with  the  light  pen  only  if  the  comput¬ 
erized  system  did  not  act  to  correct  the  altitude  deviation  within  the 
time  that  the  red  light  was  on,  i.e.,  their  role  was  to  act  as  a  backup 
in  the  event  the  system  failed.  As  noted  earlier,  corrective  action  with 
the  light  pen  was  never  necessary,  since  the  altitude  changes  always 
reverted  back  to  within-limits  values  before  the  offset  of  the  red  light. 

A  7 -minute  practice  period  followed  the  instructions  for  both  conditions. 


After  the  2r*hour  task  session,  the  subject  completed  a  second  fora  of 
the  subjective  rating  scale.  This  form  was  identical  to  the  first  except 
that  the  subject  was  asked  to  rate  eacti  item,  plus  one  additional  item 
dealing  with  task  monotony,  on  the  basis  of  how  he/she  felt  near  the 
end  of  the  test  period  just  completed. 

Results. 

A.  Comparison  of  Comput er-Contr oiled  and  Subjected- Controlled  Conditions. 
Figure  1  shows  mean  target  detection  times  across  30-minute  scoring  periods 
for  the  two  conditions.  Analysis  of  variance  applied  to  these  data  revealed 
a  significant  main  effect  for  the -four  30-minute  periods  (£(3/42)  ■  6.00, 

£  <  .01).  Although  Figure  1  suggests  detection  times  to  be  slower  under 
the  computer-controlled  condition,  analysis  of  variance  revealed  no  signi¬ 
ficant  main  effect  for  the  two  conditions  (1(1/14)  -  1.16,  p>  .05)  and 
no  significant  conditions  by  periods  interaction  (F(l/14)  ■1.41*  £  >  .05). 
Likewise,  there  was  no  evidence  of  any  difference  between  the  conditions 
in  the  total  number  of  critical  stimuli  missed.  One  subject  in  each 
condition  missed  one  stimulus;  the  rest  did  not  miss  any. 
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Figure  1.  Mean  target  detection  times  for  the  subject-controlled  and 
computer-controlled  conditions. 


Separate  t  tests  applied  to  the  rating  scale  data  revealed  no  differ¬ 
ences  (£  >  .03)  between  the  computer-controlled  and  subject-controlled 
conditions  at  either  the  beginning  or  end  of  the  experiment ^  Statements 
on  the  scales  corresponding  to  the  mean  ratings  obtained  at  the  completion 
of  the  task  period  suggested  that  the  subjects  felt  attentive,  more  tired 
than  usual,  relaxed,  indifferent  (neither  bored  nor  interested) >  but 
felt  the  task  itself  to  be  very  monotonous. 

B.  Comparison  of  ATCS  with  Non-ATCS  Subjects.  Since  there  were  no 
differences  between  the  two  experimental  conditions,  the  data  shown  in 
Figure  1  were  combined  across  groups.  These  combined  data  were  then 
compared  with  data  from  a  sample  of  non-ATCS *s.  This  latter  sample  was 
obtained  by  randomly  selecting  15  men  and  1  woman  from  a  larger  group 
of  45  subjects  used  in  a  previous  study  (9).  Subjects  in  this  earlier 
study  ranged  in  age  from  18—29,  were  all  college  students,  and  were 
tested  under  conditions  identical  to  those  of  the  subject-controlled 
condition  employed  with  the  ATCS's  in  the  present  study. 

A  comparison  of  mean  target  detection  times  of  the  ATCS  and  non-ATCS 
subjects  is  shown  in  Figure  2.  An  analysis  of  variance  of  these  data 
revealed  a  significant  difference  between  groups  (F(l/30)  «  4.43,  £  <.05), 
as  well  as  a  significant  difference  between  the  four  30-minute  periods 
(F(3/90)  -  12.70,  £  <  .01).  There  was  no  significant  interaction  effect 
(F  <  1.00). 


30-  MINUTE  PERIODS 


Figure  2.  Mean  target  detection  times  for  ATCS  and  non-ATCS  subjects. 


In  addition  to  detecting  targets  more  rapidly,  ATCS*s  also  missed 
fewer  targets.  Fourteen  of  the  ATCS's  missed  none  of  the  stimuli,  and 
the  remaining  two  only  missed  one  stimulus.  In  contrast,  five  of  the 
non-ATCS's  missed  two  to  four  stimuli,  three  missed  one  stimulus,  and 
eight  missed  none.  A  Fisher  Exact  Probability  Test  performed  on  these 
data  indicated  the  differences  between  groups  to  be  significant  (£  ■  .044). 

A  comparison  of  the  subjective  rating  scale  measures  obtained  on  both 
groups  revealed  no  differences  between  groups  on  any  of  the  presession 
measures,  but  significant  differences  on  two  postsession  measures.*  ATCS's 
rated  themselves  as  being  more  attentive  than  non-ATCS's  CtC30)  •  3.19, 

£  <  .01).  There  were  also  differences  in  rated  monotony  (but  not  boredom). 
The  ATCS's  found  the  task  to  be  "very  monotonous,"  while  non-ATCS's  rated 
the  task  as  only  "moderately  monotonous"  (t(30)  -  2.74,  £  <  .05)*  These 
differences  are  not  too  surprising  when  one  realizes  that  the  requirement 
to  maintain  high  levels  of  attentiveness  is  a  necessity  for  ATCS's  in 
the  performance  of  their  jobs.  Also,  it  would  be  expected  that  ATCS's 
would  find  our  radar  monitoring  task  quite  monotonous  relative  to  real- 
life  air  traffic  control  work  in  contemporary  systems. 

Discussion. 

Although  target  detection  times  of  ATCS's  were  generally  longer  under 
the  computer-controlled  than  under  the  subject-controlled  condition,  the 
difference  between  conditions  was  not  statistically  significant.  There 
are  several  possible  explanations  that  may  account  for  the  lack  of  signi¬ 
ficance.  The  first  and  most  obvious  one  is. that  the  experimental  manipu¬ 
lations  designed  to  increase  complacency  did  not  produce  the  desired  effect. 
The  subject-controlled  condition  used  in  the  present  study  was  essentially 
identical  to  that  used  in  most  of  our  previous  studies,  and  the  specific 
intent  of  this  condition  was  to  produce  a  task  that  simulated  a  level  of 
passive,  low  involvement  felt  to  approximate  the  task  conditions  of  highly 
automated  air  traffic  control  systems.  It  may  be  difficult,  if  not 
impossible,  to  decrease  the  level  of  involvement  beyond  that  already 
inherent  in  this  condition.  The  second  possibility  is  that  complacency 
may  indeed  have  been  greater  under  the  computer-controlled  condition,  but 
that  controllers,  because  of  their  training  and  experience,  may  be  able 
to  sustain  a  high  level  of  attention  in  spite  of  lower  task  involvement. 

If  training  and  experience  were  relevant  variables,  the  two  conditions 
compared  in  this  study  might  have  differed  significantly  had  non-ATCS's 
been  used  as  subjects. 

That  controllers  are  able  to  perform  a  radar  monitoring  task  more 
effectively  and  efficiently  than  subjects  without  training  in  air  traffic 
control  was  clearly  shown  by  the  fact  that  ATCS's  detected  targets  more 
rapidly,  made  fewer  omission  errors,  and  rated  themselves  as  more  attentive 
than  did  non-ATCS's.  The  superiority  of  ATCS's  in  speed  of  detecting 
targets  was  particularly  striking  in  that  this  superiority  was  maintained 
across  the  entire  2-hour  session. 

Despite  their  superiority  In  target  detection,  however,  ATCS's  showed 
a  decline  in  performance  during  the  last  30-mlnute  measurement  period 
thet  essentially  paralleled  the  decline  shown  for  non-ATCS's.  This 


general  decline,  known  in  the  vigilance  literature  as  the  "decrement 
function,”  has  been  found  in  all  of  our  previous  studies  using  this 
task.  Since  this  type  of  performance  decline  has  been  found  to  occur 
in  auditory  vigilance  tasks  as  well  (1) ,  it  cannot  be  attributed 
simply  to  visual  fatigue.  Rather,  it  appears  to  involve  a  decline  in 
some  central  attentional  process  which  is  known  to  promote  or  induce 
drowiness,  microsleep  episodes,  and  certain  trance-like  (highway- 
hypnotic)  conditions  (4) .  Previous  research  conducted  with  the  present 
task  has  shown  that  this  decline  in  performance  is  due  primarily  to  an 
increase  in  long  detection  times  which,  in  turn,  appear  to  result  from 
lapses  of  attention  (6,7,10).  Because  this  decline  in  performance 
(attention)  occurs  in  ATCS’s  in  spite  of  their  extensive  experience  with 
similar  displays,  our  results  suggest  the  possibility  that  controllers  in 
future  systems,  in  which  monitoring  may  be  a  major  task  requirement,  may 
show  a  like  decline  in  attentiveness  after  approximately  90  minutes  of 
continuous  system  monitoring.  The  magnitude  of  the  decline  in  detection 
efficiency  shown  in  our  study  is  not  overly  impressive  in  terms  of  the 
absolute  increase  in  detection  time,  but  if  taken  as  an  indication  of 
decreased  alertness,  ability  to  react  rapidly  and  efficiently  to  an 
emergency  condition  or  sudden  task  overload  during  this  period  of 
declining  attentiveness  could  be  substantially  impaired.  The  question 
of  how  effective  is  the  response  to  emergency  situations  during  a  period 
of  declining  alertness  will  be  addressed  specifically  in  a  laboratory 
study  currently  being  planned  using  a  simulation  of  advanced  radar  monitoring 
concepts  still  in  the  development  phase. 
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